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SPECIFICATION 

LIVE STREAMING BROADCASTING METHOD, LIVE STREAMING 
BROADCASTING APPARATUS, LIVE STREAMING BROADCASTING SYSTEM, 
PROGRAM, RECORDING MEDIUM, BROADCASTING METHOD AND 
BROADCASTING APPARATUS 

TECHNICAL FIELD 

[0001] The present invention relates to a live streaming broadcasting mehod, a live 
streaming broadcasting apparatus, a live streaming broadcasting system, 
programs, a recording medium, a broadcasting method and broadcasting 
apparatus. 

BACKGROUND ART 

[0002] There has been carried out conventionally an internet live broadcasting for 
broadcasting images (pictures) and voices to clients through a network such as 
internet, that is, a live streaming broadcasting. 

[0003] In order that clients audit live streaming broadcasting, for example, browser is 
started by a terminal for auditory to get access to a home page of a broadcasting 
presenter (a broadcaster), and data of the broadcasting contents is received by 
the terminal for auditory. The data received by the terminal for auditory is 
converted into a streaming file by the decoding process in a streaming player 
(including a streaming decoder) incorporated in the terminal for auditory in 
advance so that an image out of the broadcasting contents is displayed on the 
display screen of the terminal for auditory, and the voice is output from a 
speaker. Thereby, the clients are able to audit the broadcasting contents. 

[0004] It is to be noted that the terminal for auditory is, for example, a general purpose 
PC (Personal Computer). Further, the streaming player is a streaming player 
incorporated into a general purpose browser, or an exclusive use streaming 
player, both of which are constructed within the terminal for auditory by 
installing a program (software) on the terminal for auditory. 

[0005] On the other hand, in order that the broadcaster performs the live streaming 
broadcasting, a broadcasting program is started by a broadcasting terminal, 
while voice data are input into the broadcasting terminal from, for example, 
camera image data and mike. And, these data are subjected to 
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encode-processing in accordance with the broadcasting program started to 
output them to the network. 

[0006] It is noted that the broadcasting terminal is, for example, a general purpose PC, 
And, the broadcasting program is a general purpose program (software) 
including a function as a streaming encoder. 

[0007] FIG 14 shows a flow of the live streaming broadcasting as described above. 

[0008] As shown in FIG 14, on the broadcaster side, image data (animations) from a 
camera 101 and voice data from a mike 102 are encode-processed and 
converted into a streaming file, which is continuously output to a network 104. 
Broadcasting data to be output are input into a streaming server 106. 

[0009] Further, in a terminal for auditory 105 on the client side, a browser 105a is 
started so that broadcasting data from the broadcaster are continuously received 
through a network 104 from a streaming server 106, the received broadcasting 
data are decode-processed by a streaming player (streaming decoder) 105b 
within the terminal for auditory 105 to continuously carry out image display 
and voice output. Thereby, on the auditor side, broadcasting through the 
network 104 can be audited (on live) at real time. 

[0010] As the technique in connection with the live streaming broadcasting as 
described, there is a technique of Patent Reference 1, for example. 

[0011] Incidentally, in the conventional live streaming broadcasting for carrying out 
editing mainly using a single apparatus, for example, such as a general purpose 
PC, the broadcasting mode for broadcasting images and voices without 
applying any process thereto is a mainstream, and the expression thereof 
involved a great difference as compared with a radio broadcasting. That is, 
conventionally, it has been impossible, for example, to synthesize a plurality of 
camera images, to insert a telop, £nd to perform processes for synthesizing 
video images (such as alpha blend process, lay over laying process). 
[0012] Concretely, for example, according to the editing function provided in 
T WindowsMedia Encoder f goods name J J which is a software for the 
streaming encode made by Microsoft Inc., only one camera source is selected, 
and a plurality of camera images cannot be displayed simultaneously. 
[0013] Further, in case of processing images, for example, as disclosed in Patent 
Reference 1, there is merely present such that separate images are to be merely 
displayed on a plurality of display regions, respectively, within the display 
screen on the clients side. 
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[0014] In order that conventionally, for example, synthesizing a plurality of camera 
images, inserting a telop, and performing processes for synthesizing video 
images (such as alpha blend process, lay over laying process). are carried out, it 
was necessary to use a broadcasting system 200 provided with many 
broadcasting apparatuses in addition to PC200, as shown in FIG 15, for 
example. 

[0015] That is, the broadcasting system 200 shown in FIG 15 is provided, for editing 
images, with for example, PC201 having display data such as a telop stored 
therein, a down converter 202, a plurality of video decks 203 for regenerating a 
video tape, a switcher 204 for selecting one out of these image data, a 
confirming monitor 205, a plurality of cameras 206, a switcher 207 for 
selecting one out of image data from the plurality of cameras 206, a confirming 
monitor 208, a video mixer (which performs alpha blend process, lay 
overlaying process, etc) for synthesizing image data from the switches 204, 
207), and a monitor 210 for confirming image data after synthesized by the 
video mixer 209. 

[0016] Further, for editing voices, there are provided, for example, with a sampler 211 ■ 
for sampling effect sound, an effecter for applying effect process to effect 
sound, a mike 213, a player 214 such as CD player, MIDI apparatus 215 for 
regenerating a MIDI file, voice apparatus 216 for line-inputting voice data, a 
mixer 217 for mixing these voice data, and a monitor 218 for monitoring voice 
data after mixing by the mixer 217. 

[0017] Further, the PC 220 is provided with a video capture 221 for receiving image 
data from the video mixer 209. a sound card 222 for receiving voice data from 
the mixer 217, and a stream encoder (streaming encoder) 223 fir 
encode-processing voice data from £he sound card 222 and image data from the 
video capture 221 into streaming broadcasting to output them to the network 
104. 

Patent Reference 1: Japanese Patent Laid-Open No. 2003-125339 Publication 
( P 2-p6) 

DISCLOSURE OF INVENION 
Problem to be solved by the Invention 
[0018] In the case of the conventional live streaming broadcasting for performing 
editing mainly using a single apparatus such as a general purpose PC, there was 
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a great different in its image and voice from radio broadcasting, as described 
above. 

[0019] Further, concretely, for example, in the case of using T Windows Media 
Encoder Tgoods nameJJ, for image- switching between cameras, it is necessary 
to take the procedure for starting a separate camera after finishing operation of 
the camera selected originally, thus posing a problem that takes time for 
switching. 

[0020] Further, in the case of using a broadcasting system 200 provided with many 
broadcasting devices as shown in FIG 15, there needs costs for materials, and it 
takes time for installation of materials as well as connection. In addition, in 
case of the broadcasting system 200, there poses a problem discussed below. 

[0021] FIG 16 is a flowchart showing a flow of processes carried out particularly in a 
switcher 207 and a video mixer 209, out of various broadcasting devices shown 
in FIG 15. 

[0022] As shown in FIG 16, in the switcher 207, image data are input from a plurality 
of cameras 206 (Step S101), D/A conversion is carried out with respect to these 
image data (Step SI 02), and subsequently, image data from a camera 21 that is 
selected by operation of a broadcaster out of the image data are selected (Step 

51 03) . Then, the selected image data is subjected to D/A conversion (Step 

51 04) to output it from the switcher (Step SI 05). 

[0023] Further, in the video mixer 209, image data from the switcher 207 are 
respectively input (SI 06), which are subjected to A/D conversion with respect 
to the image data (Step SI 07). Then, the image data after A/D conversion are 
synthesized (Step S108), and the image data after synthesized are subjected to 
D/A conversion to output it to PC 220 from the video mixer 209. 

[0024] That is, in the case of the broadcasting system 200, since the synthesizing 
process (Step SI 04) is carried out, it is necessary, as shown in FIG 16, to carry 
out output and input of image data (Step SI 05 and Step SI 06), and it is also 
necessary to repeat A/D conversion (Step SI 02 and Step SI 07) and D/A 
conversion (Step SI 04 and Step SI 09), resulting in a lot of wastes in process. 
Moreover, input and output, and D/A conversion are repeated, thus posing a 
problem of increasing possibility that noses are produced in image data. 

[0025] Further, for inserting a telop in the conventional live streaming broadcasting, it 
is necessary to prepare display data for a telop in advance to store it in PC201, 
which is troublesome, and failing to correspond thereto in the case where a 
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telop becomes necessary suddenly. 
[0026] Further, in the conventional live streaming broadcasting, it is merely that one 

dimensional broadcasting from a single streaming server 106 can be audited. 

Therefore, it was not possible to audit multi-dimensional broadcasting from a 

plurality of streaming servers 106. 
[0027] In the live streaming broadcasting, there is a problem that for the convenience' 

sake of data amount that can be processed, it is difficult to use excessively large 

image data for broadcasting. Therefore, it has been desired to be data amount 

processed as small as possible, as well as broadcasting contents excellent in 

expression. 

[0028] Furthermore, also in various broadcastings not limited to the live streaming 
broadcasting, a broadcasting method in novel expression that not found in prior 
art has been desired. 

[0029] The present invention has been accomplished in order to solve the problems as 
noted above, and has its object to provide a live streaming broadcasting method, 
a live streaming broadcasting apparatus, a live streaming broadcasting system, 
programs, a recording medium, a broadcasting method and a broadcasting 
apparatus, which realize broadcasting in high expression at low cost, or in 
novel expression not obtained so far. 

Means for solving the problem 

[0030] For solving the aforesaid problems, the live streaming broadcasting method of 
the present invention is a live streaming broadcasting method for carrying out a 
live broadcasting through a network, characterized in that, while inputting a 
plurality of camera image data, synthesized image data obtained by 
synthesizing process for synthesiziqg a plurality of camera image data during 
inputting are output for auditory by clients. 

[0031] Further, the live streaming broadcasting method of the present invention is a 
live streaming broadcasting method for carrying out a live broadcasting through 
a network, characterized in that, while receiving the other live streaming 
broadcasting, image data of the live streaming broadcasting during receiving 
are output for auditory by clients. 

[0032] In this case, preferably, while receiving a plurality of said other live streaming 
broadcasting, synthesized image data obtained by the synthesizing process for 
synthesizing image data of said plurality of live streaming broadcasting during 
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receiving are output for auditory by clients. 

[0033] Further, the live streaming broadcasting method of the present invention is a 
live streaming broadcasting method for carrying out a live broadcasting through 
a network, characterized in that, while inputting camera image data, 
synthesized image data obtained by the synthesizing process for synthesizing 
the other image data to the camera image data during inputting are output to 
said network for auditory by clients. 

[0034] Further, the live streaming broadcasting method of the present invention is 
characterized in that at least either one out of static image data and video image 
data is included in said other image data. 

[0035] Further, the live streaming broadcasting method of the present invention is 
characterized in that text display data input by operation during broadcasting 
are included in said other image data. 

[0036] Further, the live streaming broadcasting method of the present invention is 
characterized in that image data produced on the basis of designated 
information which is for designating image display but not image data are 
included in said other image data. 

[0037] Further, the live streaming broadcasting method of the present invention is 
characterized in that plug-in data are included in said other image data. 

[0038] Further, the live streaming broadcasting method of the present invention is 
characterized in that said synthesizing process is alpha blend process or 
picture-in-picture process. 

[0039] Further, the live streaming broadcasting method of the present invention is a 
live streaming broadcasting method for carrying out a live broadcasting through 
a network, characterized in that text display data input by operation during 
broadcasting are output to said network for auditory by clients. 

[0040] Further, Further, the live streaming broadcasting method of the present 
invention is a live streaming broadcasting method for carrying out a live 
broadcasting through a network, characterized in that image data the live 
streaming broadcasting method of the present invention is characterized in that 
image data produced on the basis of designated information which is for 
designating image display but not image data are output to said network for 
auditory by clients. 

[0041] Further, Further, the live streaming broadcasting method of the present 
invention is a live streaming broadcasting method for carrying out a live 
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broadcasting through a network, characterized in that plug-in data are output to 
said network for auditory by clients. 
[0042] Further, the live streaming broadcasting method of the present invention is a 
live streaming broadcasting method for carrying out a live broadcasting through 
a network, characterized in that link-addressed information of browser on the 
broadcaster side is output as a script, and the link address of browser on the 
clients side is designated on the basis of the script of said link addressed 
information to thereby switch the link address synchronously with the 
broadcaster side. 

[0043] Further, the live streaming broadcasting method of the present invention is a 
live streaming broadcasting method for carrying out a live broadcasting through 
a network, characterized in that position information displayed on the browser 
on the broadcaster side is output as a script, and a display position of a pointer 
on the browser on the clients side is designated on the basis of the script of said 
position information to thereby associate a display position of the pointer on the 
clients side with the broadcaster side. 

[0044] Further, the live streaming broadcasting method of the present invention is a ■ 
live streaming broadcasting method for carrying out a live broadcasting through 
a network, characterized in that image data of image depicted by operation of 
the broadcaster on the browser on the broadcaster side are output to said 
network for auditory by clients. 

[0045] Further, the live streaming broadcasting method of the present invention is 
characterized in that said image data of image depicted by operation of the 
broadcaster are synthesized with animation image data to output them to said 
network. 

[0046] Further, the live streaming broadcasting apparatus of the present invention is a 
live streaming broadcasting apparatus for carrying out a live broadcasting 
method through a network, characterized by comprising synthesizing 
processing means for executing the synthesizing process in an either live 
streaming method of the present invention, and output means for executing said 
output to said network. 

[0047] Further, the live streaming broadcasting apparatus of the present invention is a 
live streaming broadcasting apparatus for carrying out a live broadcasting 
method through a network, characterized by comprising receiving means for 
receiving the other live streaming broadcasting through said network, and 
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output means for outputting image data of said live streaming broadcasting 
during receiving to said network for auditory by clients. 

[0048] Further, the live streaming broadcasting apparatus of the present invention is a 
live streaming broadcasting apparatus for carrying out a live broadcasting 
method through a network, characterized by comprising output means for 
outputting text display data input by operation during broadcasting to said 
network for auditory by clients. 

[0049] Further, the live streaming broadcasting apparatus of the present invention is a 
live streaming broadcasting apparatus for carrying out a live broadcasting 
method through a network, characterized by comprising output means for 
outputting image data produced on the basis of designated information which is 
for designating image display but not image data to said network for auditory 
by clients. 

[0050] Further, the live streaming broadcasting apparatus of the present invention is a 
live streaming broadcasting apparatus for carrying out a live broadcasting 
method through a network, characterized by comprising output means for 
outputting plug-in data to said network for auditory by clients. 

[0051] Further, the live streaming broadcasting apparatus of the present invention is a 
live streaming broadcasting apparatus for carrying out a live broadcasting 
method through a network, characterized link-addressed information of 
browser on the broadcaster side is output as a script, and the link address of 
browser on the clients side is designated on the basis of the script of said link 
addressed information to thereby switch the link address synchronously with 
the broadcaster side. 

[0052] Further, the live streaming broadcasting apparatus of the present invention is a 
live streaming broadcasting apparatus for carrying out a live broadcasting 
through a network, characterized in that position information displayed on the 
browser on the broadcaster side is output as a script, and a display position of a 
pointer on the browser on the clients side is designated on the basis of the script 
of said position information to thereby associate a display position of the 
pointer on the clients side with the broadcaster side. 

[0053] Further, the live streaming broadcasting apparatus of the present invention is a 
live streaming broadcasting apparatus for carrying out a live broadcasting 
through a network, characterized by comprising output means for outputting 
image data of image depicted by operation of the broadcaster on the browser on 
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the broadcaster side are output to said network for auditory by clients. 

[0054] Further, the live streaming broadcasting apparatus of the present invention is 
characterized by comprising synthesizing means for synthesizing said image 
data of image depicted by operation of the broadcaster with animation image 
data, and said output means outputs image data after synthesized by said 
synthesizing means to said network. 

[0055] Further, the live streaming broadcasting system of the present invention is 
characterized by comprising a live streaming broadcasting apparatus of the 
present invention, and a streaming server for delivering image data output from 
said live streaming broadcasting apparatus to clients. 

[0056] Further, the program of the present invention is a program that can be read by a 
computer, and plural cameral synthesizing process for synthesizing a plurality 
of image data input in apparatus provided with said computer is allowed to be 
executed by said computer, characterized in that switching process for selecting 
camera image data in order to selectively apply a suitable plurality of camera 
image data out of three camera image data or more input in said apparatus, and 
output process for outputting synthesized image data produced by said plural 
cameral synthesizing process from said apparatus are allowed to be executed in 
that order by said computer. 

[0057] Further, the program of the present invention is a program that can be read by a 
computer, characterized in that synthesizing process in the streaming 
broadcasting method of the present invention and said output to said network 
are allowed to be executed by said computer. 

[0058] Further, the program of the present invention is a program that can be read by a 
computer, characterized in that process for receiving a live streaming 
broadcasting through a network, ar\d process for outputting the live streaming 
broadcasting during receiving to said network for auditory by clients are 
allowed to be executed by said computer. 

[0059] Further, the program of the present invention is a program that can be read by a 
computer, and the live streaming broadcasting through a network is allowed to 
be executed by said computer, characterized in that process for outputting text 
display data input by operation during broadcasting of the live streaming 
broadcasting to said network for auditory by clients is allowed to be executed 
by said computer. 

[0060] Further, the program of the present invention is a program that can be read by a 
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computer, and the live streaming broadcasting through a network is allowed to 
be executed by said computer, characterized in that process for outputting 
image data produced on the basis of designated information that is for image 
display designation but not image data to said network for auditory by clients is 
allowed to be executed by said computer. 
[0061] Further, the program of the present invention is a program that can be read by a 
computer, and the live streaming broadcasting through a network is allowed to 
be executed by said computer, characterized in that process for outputting 
plug-in data to said network for auditory by clients is allowed to be executed by 
said computer. 

[0062] Further, the program of the present invention is a program that can be read by a 
computer, and the live streaming broadcasting through a network is allowed to 
be executed by said computer, characterized in that process for outputting link 
address information of browser on the broadcaster side, designating a link 
address of browser on the clients side on the basis of a script of said link 
address information, and thereby synchronously switching the link address on 
the clients side to the broadcaster side. 

[0063] Further, the program of the present invention is a program that can be read by a 
computer, and the live streaming broadcasting through a network is allowed to 
be executed by said computer, characterized in that process for outputting, as a 
script, position information of a pointer displayed on browser on the 
broadcaster side, designating a display position of the pointer on the browser on 
the clients side on the basis off said position information, and thereby 
associating the display position of the pointer on the clients side with the 
broadcaster side is allowed to be executed by said computer. 

[0064] Further, the program of the present invention is a program that can be read by a 
computer, and the live streaming broadcasting through a network is allowed to 
be executed by said computer, characterized in that process for outputting 
image data of image depicted by operation of the broadcaster on the browser on 
the broadcaster said network for auditory by clients is allowed to be executed 
by said computer. 

[0065] The program of the present invention is a program that can be read by a 
computer is characterized in that process for outputting image data including 
plug-in data to a broadcasting network for auditory by clients is allowed to be 
executed by said computer. 
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[0066] Further, the recoding medium of the present invention is characterized in that 

the program of the present invention is recorded. 
[0067] Further, the broadcasting method of the present invention is characterized in 

that image data including plug-in data are output to the broadcasting network 

for auditory by clients. 
[0068] Further, the broadcasting apparatus of the present invention is characterized by 

comprising output means for outputting image data including plug-in data to 

the broadcasting network for auditory by clients. 

EFFECT OF THE INVENTION 
[0069] According to the present invention, broadcasting which is high in expression at 

low cost can be executed. 
[0070] Or, broadcasting in novel expression that cannot be obtains so far can be 

realized. 

BEST MODE FOR EMBODYING THE INVENTION 
[0071] The embodiments according to the present invention will be described * 

hereinafter with reference to the drawings. 
[0072] [First Embodiment] 

FIG 1 is the whole block view showing various structural elements for 

realizing the streaming broadcasting method according to the present 

embodiment. 

[0073] As shown in FIG 1, in the streaming broadcasting method according to the 
present embodiment, in the editing apparatus (streaming broadcasting 
apparatus) 1 on the broadcaster side, while producing image data and voice 
data by editing process, said image A data and voice data to be produced, that is, 
image data and voice data after editing process are continuously output, as 
broadcasting data, to a streaming server 3 through a networks. Here, the 
streaming server 3 at output address is designated in advance by input of IP 
(Internet protocol) by the broadcaster or selecting work. Further, as the network 
2, there can be mentioned, for example, internet, LAN, communication network 
of portable information terminal, and the like. Further, as the editing apparatus 
1, there can be, for example, a general purpose PC (Personal Computer). 

[0074] On the other hand, in a terminal for auditory 4 on the clients side, while 
continuously receiving image data and voice data (broadcasting data) from the 
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streaming server 3 through the network 2, they are displayed on a display 
portion of the terminal for auditory 4, and output from a speaker of the terminal 
for auditory 4. 

[0075] Thereby, the clients are able to audit images based on image data from the 
broadcaster side continuously and at real time through the network 2. 

[0076] It is noted that as the terminal for auditory 4, there can be mentioned, for 
example, a portable information terminal apparatus such as PDA or as a 
portable telephone, in addition to a general purpose PC. 

[0077] More concretely, at the time of auditory, for example, the clients get access to a 
home page prepared in advance by the broadcaster side and click 
fbroadcasting start buttonj within the home page to thereby enable (display 
and voice output) of broadcasting. Or, broadcasting is started merely by getting 
access to the home page on the broadcaster side. At this time, a streaming layer 
82 (including a streaming decoder) is started so that an image display of 
broadcasting is done within a player screen, or an image display is done within 
a screen of browser 81. As described, for getting access to the home page to do 
auditing, the broadcaster stores data of the home page in advance in a server (a 
server for home page separately from the streaming server 3) 5. 

[0078] It is noted that the other streaming server 6 (FIG 1) for broadcasting is a 
streaming server (for example, for the other broadcaster) in order to perform 
live streaming broadcasting by image data output from apparatus except the 
editing apparatus 1 . 

[0079] In the foregoing, transmit/receive of broadcasting data (transmit/receive 
between editing apparatus 1-*- streaming server 3, and between streaming server 
3 -^terminal for auditory 4) is carried out by designating transmit/receive ends 
by IP (Internet Protocol). 

[0080] FIG. 2 is a block diagram showing the editing apparatus 1 and its peripheral 
apparatus. 

[0081] As shown in FIG 2, camera image data from a plurality (for example, six) of 
cameras 21 are input in the editing apparatus on the broadcaster side. It is noted 
that the camera 21 may be a camera for outputting camera image data as digital 
data. It may be one for outputting as analog data. In case of using the camera 21 
for outputting as analog data, in the editing apparatus 1, editing apparatus 
(described later) is applied to camera image data to be input after A/D 
conversion. 
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[0082] Further, in the editing apparatus 1, voice data from a kike is input, or voice data 
from external voice data outputting apparatus 23 is line-input. It is noted that 
the external voice data outputting apparatus 23 may be, for example, CD 
(Compact Disk) player or MD (Mini Desk) player. 

[0083] In addition, a video card 24 for processing image data, sound cards 25, 2 for 
processing voice data are inserted into the editing apparatus 1. 

[0084] Further, for example, a head phone (second sound device) as a voice monitor is 
connected to the editing apparatus 1 . 

[0085] Further, the editing apparatus is provided with a display portion 12 for 
displaying an operation screen Gl (FIG 6) including a display region of image 
data before editing (source image data) and images (images o be broadcast) 
after editing, a speaker (first sound device) 13 for outputting, for example, 
voice after editing, an operation portion 14 for carrying out editing operation, a 
clock portion 15 for carrying out time checking of time and time measuring, 
and a control portion 1 1 for carrying out editing process or display control of 
the display portion 12 according to the operation with respect to the operation 
portion 14, being connected to the network 2. 

[0086] Among them, the display portion 12 comprises, for example. A liquid crystal 
display device or display device of a cathode-ray tube system. Outputting of 
display data (image data) to the display portion 12 is carried out through a 
video buffer 24a of the video card 24, for example. 

[0087] Further, outputting of voice data to the speaker 13 is carried out through a 
sound buffer 25a of the sound card 25, for example. 

[0088] Further, the operation portion 14 is constructed by being provided with a 
keyboard 14a and a mouse 14b, for example c 

[0089] Further, the control portion 11 is, A as shown in FIG 3, constructed by being 
provided with CPU (Central Processing Unit), ROM (Read Only Memory) lib, 
RAM (Random Access Memory) 11c and an input/output interface lid, for 
example. 

[0090] Among them, CPU 11a is provided with an operating portion and a control 
portion, and programs stored in the ROM lib are executed to thereby perform 
editing process of broadcasting data (image data and voice data), output process 
of broadcasting data to the network 2, output process of voice data to the head 
phone 27, and operation control of the display portion 12 and the speaker 13. 

[0091] In the ROM (recording medium) lib are stored programs for operation and 
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control, and data used for exiting. 

[0092] Concretely, the programs stored in ROM lib include, for example, an editing 
program 31, a streaming decoder program 32, a streaming encoder program 33, 
and a video decoder program 38. 

[0093] Further, data for editing stored in ROM lib include, for example, static image 
data 34, video image data 35, effect sound data 36 and music data 37. Among 
them, the static image data 34 is, for example, JPEG; the video image data 35 is, 
for example, AVI or mpeg; the effect sound data 36 is, for example, a WAVE 
file; and the music data 37 is, for example, a WAVE file, mp3, WMA or MIDI. 

[0094] RAM 11c is provided with a work region for CPU 11a. In editing, in 
accordance with the editing program 31, RAM 1 lc is formed, for example, with 
a capture window 41, a picture buffer (for example, two picture buffers 
comprising a first picture buffer 42 and a second picture buffer), and a main 
picture buffer 44 for temporarily storing image data after all image synthesizing 
processes have been finished. It is noted that the number of the picture buffers 
is the number corresponding to that of image data to be synthesized. That is, if 
the number of image data to be synthesized is 3 or more, the number of picture 
buffers is also 3 or more. 

[0095] In the above-described structure, a live streaming broadcasting system 50 
according to the present embodiment is constructed by the editing apparatus 1, 
camera 21, mike 22, video card 24, sound cards 25, 26, voice apparatus 23, 
head phone 27, streaming server 3 and server 5. 

[0096] In the following, various processes carried out by CPU 11a on the basis of the 
execution of the programs will be described. 

[0097] <Process based on the execution of the video decider program38> 

CPU 11a performs the process for^decoding video image data 35 as a video 
decoder 45 (FIG. 4) (video decoder process). 

[0098] <Process based on the execution of the video decider program32> 

CPU 11a performs the process for decoding video image data 35 as a video 
decoder 45 (FIG 4) (video decoder process). 

[0099] <Process based on the execution of the streaming decider program33> 

CPU 11a performs the process for decoding live streaming broadcasting data 
received from the other streaming server 6 through the network 2 as a 
streaming decoder 46 (FIG 4) (streaming decoder process). 

[0100] <Process based on the execution of an editing program> 
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In the following, there are listed processes performed by CPU 11a based on the 
execution of the editing program. 

[0101] "Capture window producing process" 

Process for producing capture windows 41 (in case of the present embodiment, 
concretely, for example, 6 capture windows 41) so as to correspond to a 
plurality (in case of the present embodiment, concretely, for example, 6) of 
cameras 21 in 1:1. 

[0102] "First switching control process" 

Process for selecting data for storage to a first buffer picture buffer 42 and a 
second picture buffer 43 out of camera image data received in the capture 
window 41. 

[0103] However, in case where only (one) camera image data from one camera 21 is 
used for editing, the one camera image data is selected for storage in the first 
picture buffer 42. 

[0104] "Picture buffer storage process" 

Process in which camera storage data selected for storage in the first picture 
buffer by switching control, and camera storage data selected for storage in the 
second picture buffer are temporarily stored in the first picture buffer 42 and 
the second picture buffer 43, respectively. 

[0105] In case where only (one) camera image from one camera 21 is used for exiting, 
the one camera image data is stored in the first picture buffer 42, and camera 
image data is not stored in the second picture buffer. 

[0106] "Plural camera image synthesizing process" 

Plural camera image synthesizing process for synthesizing camera image data 
stored in the first and second picture buffers 42, 43 to produce synthesized 
image data (Step S2 in FIG 4). This plural camera image synthesizing process, 
concretely, include, for example, alpha blend process and picture-in-picture 
process. Among them, the alpha blend process is a process for synthesizing a 
plurality of images in a half-transparent state to synthesize them. For example, 
by using the alpha blend process such that while one transparency of image is 
gradually made higher, the other transparent of image is gradually made lower 
to thereby enable switching between the cameras 21 without difference. Further, 
the picture-in-picture process is a process for displaying the other image in one 
image on a small window, which is able to display images of a plurality of 
cameras 21 simultaneously. 
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[0107] It is noted that in case where camera image data selected by the first switching 
control process (Step SI) is only one, the plural camera image synthesizing 
process is not executed. 

[0108] "Process for producing data for a telop" 

Process for producing display data, as a telop, of text data input by operation of 
the keyboard 14a to insert (synthesize) in camera image at real time (Step S3 in 
FIG4). 

[0109] "Process for producing display data for information" 

Process for producing, as display data for information, display data on the basis 
of information applied to display designation (for example, time, camera 
position and lap time (in race), score in sport game relay) (Step S3 in FIG 4) 

[0110] "Plug-in data producing process" 

Process for producing plug-in data (for example, FLASH animation) (Step S5 
in FIG4). 

[0111] "Static image data obtaining process" 

Process for obtaining static image data 35 selected. 

[0112] "Second switching control process 

Process for selecting, for synthesizing process (Step S7 in FIG 4, described 
later), at least one of image data selected by data producing process for a telop 
(Step S3 in FIG 4), data producing process for information (Step S4 in FIG 4), 
plug-in data processing process (Step S6 in FIG 4), static image data 
obtaining process, video decoder process and trimming decoder process. 

[0113] "Image synthesizing process" 

Process for further synthesizing (Step S7 in FIG 4) image data selected by 
second switching control process, synthesized image data produced by plural 
camera image synthesizing process (Step S2). The image data produced by 
this image synthesizing process is display data of image which is the same as 
that is broadcasted. 

[0114] It is noted that where the plural camera image synthesizing process is not 
executed, In this image synthesizing process, there is carried out process for 
synthesizing camera image data from the first picture buffer 42, and image data 
selected by the second switching control process. 

[0115] "Main picture buffer storage process" 

Process for temporarily storing image data produced by the image synthesizing 
process (Step S7) in the main picture buffer 44. 
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[0116] Video buffer storage process". 

Process for storing image data from the main picture buffer 44 in a video buffer 
24a of the video card 24. 
[0117] "Primary buffer storage process for effect sound" 

Process for storing effect sound data 36 selected in a primary buffer 51 for 
effect sound (FIG 5). 
[0118] "Sound effect process" 

Process (Step Sll in FIG 5) for applying sound effect to effect sound data 36 
selected. 

[0119] "Secondary buffer storage process for effect sound" 

Process for collecting effect sound data 36 after sound effect process (Step 

SI 1) to store them in a secondary buffer 52 for effect sound. 
[0120] "Music data decode process" 

Process for decoding selected music data 37 as a decoder 53. 
[0121] "Music data mixer process" 

Process for mixing a plurality of music data 37 decoded by a decoder 53. 
[0122] "Mixer process" 

Process for mixing effect sound data 36 from a secondary buffer 52 for effect 

sound, voice data from voice apparatus 23, voice data from a mike 22, and 

music data after music data mixer process to thereby produce the dame voice 

data as that is broadcasted (Step S13 in FIG 5). 
[0123] "First sound buffer storage process" 

Process for temporarily storing voice data after mixer process (Step SI 3) in a 

sound buffer 25a of the sound card 25. 
[0124] "First sound device output process" 

Process for outputting music data ^tored in the sound buffer 25a to a speaker 

13 as a first sound device. 
[0125] "Mixer process for mixer" 

Process for mixing music data selected for monitor (Step S14 in FIG 4). 
[0126] "Second sound buffer storage process" 

Process for temporarily storing music data after the mixer process for monitor 

(Step SI 4) in a sound buffer 26a of the sound card 26. 
[0127] "Second sound device output process" 

Process for outputting music data stored in the sound buffer 26a in a head 

phone 27 as a second sound device. 



18 



[0128] "Operating screen display process" 

Process for displaying an operation screen Gl of FIG 1 on the display screen 
of the display portion 12. 

[0129] Here, the function of various display regions formed in the operation screen Gl 
and operating buttons will be described referring to Fig. 6. 

[0130] That is, the operation screen Gl is formed with a display region 61 for carrying 
out image display on the basis of camera image data from any of camera 21 
selected out of a plurality of cameras 21, an operating button 62 for switching 
camera images display on the display region 61, a display region 63 for 
displaying an image (image based on image data after image synthesizing 
process of Step S7) that is the same as that to be broadcasted or selected plug-in 
data (at the time of selection), a display region 64 for displaying an operating 
window for executing various functions such as telop input, an operating button 
65 for switching various functions using the display region 64, a cross fader 
operating portion 68 for images for carrying out switching between cameras 21, 
an operating button 69 for adding image effects such as picture-in picture, telop 
insertion, static image synthesizing and the like, an operating button 71 for 
selecting effect sound data 36, a display region 72 for displaying a list of 
selection candidate of music data 37, a speaker 13, and a cross fader operating 
portion 73 for voices for adjusting sound amount of he head phone 27. 

[0131] It is noted that the operating buttons 62, 65, 67, 69, 71 can be operated by 
clicking them using a mouse 14b, and the cross fader for images 68 and the 
cross fader operating portion for voices 73 can be operated by dragging them 
using a mike 14b. 

[0132] Further, the image data of images displayed on the display region 11 are input 
in the display portion 12 through th§ video buffer 24a of the video card 24 from 
any of the capture windows 41 selected, and displayed on the basis of the 
image data (In FIG 4, for the simple sake, the video card 24 in a signal channel 
to the display portion 12 from the capture window 41 is omitted.). 

[0133] In the following, an example of concrete operation will be described. 

[0134] <First Operating Example> 

In the first operating example, a description will be made of the case where 
while inputting one image data from one camera 21, synthesized image data 
obtained by synthesizing other image data to said camera image data during 
inputting is output to a network 21 for auditory by clients. 
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[0135] In this case, in a first switching control (Step SI), only the camera image data 
received in any one of capture window 41 is selected for storage to a first 
picture buffer 42. Plural camera image synthesizing process S2 is not applied to 
camera image data read from the first picture buffer 42, but said camera image 
data is applied without modification to image synthesizing process (Step S7). 

[0136] On the other hand, in a second switching control (Step S6), at least one image 
data obtained by data producing process for a telop (Step S3), display data 
producing process for information (Step S4), plug-in producing process (Step 
S5), static image obtaining process, video decoder process, and streaming 
decoder process. 

[0137] Further, in image synthesizing process (Step S7), image data selected by a 
second switching control (Step S6), and image data from a first picture buffer 
42 are synthesized. Thereby, data for display of the same image as that is 
broadcasted is produced. 

[0138] Image data after image synthesizing process is stored in a main picture buffer 
44, and further stored in a video buffer 24a. 

[0139] Image data of the video buffer 24a are output to a display portion 12 for • 
monitor, and applied to display in a display region 63 (FIG 6), and are output 
also for encode process by a streaming encoder 47. 

[0140] On the other hand, voice data from voice apparatus or a mike 22, effect sound 
data 36 having sound effect process applied, and at least any of voice data out 
of music data 37 to which decode process is applied are made to the same voice 
data as that are broadcasted by mixer process (Step SI 3), after which they are 
output for encode process by a streaming encoder 47. 

[0141] In the streaming encoder 47, image data from a video buffer 24a and voice data 
from a sound buffer 25a are encoded for streaming broadcasting, and data 
(broadcasting data) after encoding are output continuously to a network 2. 

[0142] Also, on the clients side, a browser 81 (FIG 1) is started in a terminal for 
auditory 4 to get access to a home page of the broadcaster, and display data of 
the home page is obtained by a server 5 (server for the home page of the 
broadcaster). 

[0143] And, screen display of the home page is started, or ["broadcasting start buttonj 
formed in a display screen of the home page is clicked to thereby start live 
streaming broadcasting. At that time, in a terminal for auditory 4, a streaming 
player (streaming decoder) 82 is started. The streaming player 82 performs 
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image display based on image data received continuously from a streaming 
server 43, and outputs voice data based on voice data received continuously 
from a streaming server 3 from a speaker of a terminal for auditory 4. 
Thereby, clients are able to audit live streaming broadcasting. 

[0144] As described above, according to the first operating example, the clients are 
able to audit images based on synthesized data obtained by synthesizing other 
image data with camera image data. 

[0145] <Second Operating Example> 

In the second operating example, a description will be made of the case where 
while inputting a plurality of image data, synthesized image data obtained by 
synthesizing a plurality of image data during inputting is output to a network 
for auditory by clients. 

[0146] In this case, in a first switching control (Step SI), camera image data received 
in any one of capture window 41, and camera image data received in any of 
other capture window 42 are selected for storage to a first picture buffer 42, and 
for storage to a second picture buffer 43, respectively. Further, plural camera 
image synthesizing process (Step S2) is applied to the camera image data read 
from the first and second picture buffers 42, 43 to thereby produce synthesized 
image data. 

[0147] Further, in this case, in a second switching control (Step S6), at least any one of 
image data may be selected, similarly to the case of the first operating example, 
or any one of image data may not be selected. 

[0148] In image synthesizing process (Step S7), in the case of selecting anyone of 
image data in a second switching control, there is carried out process for 
synthesizing the selected image data, and synthesized image data after plural 
camera image synthesized process On the other hand, in the case of not 
selecting any of image data in the second switching control, the image 
synthesizing process (S7) is not carried out, but synthesized image data after 
plural camera image synthesizing process is stored without modification in a 
main picture buffer 44. 

[0149] Also, in the second operating example, voice process and thereafter image 
process are similar to that of the first operating example. 

[0150] Out of operation in the second operating example, process from image data 
input from camera 21 to plurality camera image synthesizing process (Step S2) 
will be described with reference to a flowchart of FIG 7. 
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[0151] First, image data are input from camera 21 and received in capture window 41 
(Step SI 5). It is noted that where image data from camera 21 is analog data, in 
Step (SI 5), A/D conversion is applied to image data before receipt into the 
capture window 41. 

[0152] Next, a first switching control process (Step SI) is applied to each image data. 

[0153] Next, camera image data selected in the first switching control process are 
stored in the first and second picture buffers 42, 43 (Steps SI 6, SI 7). 

[0154] Next, the plural camera image synthesizing process (Step S2) is applied to 
image data stored in the first and second picture buffers 42, 43. 

[0155] Further, image data after plural camera image synthesizing process are output to 
a network 2 after having been applied with encode process by a streaming 
encoder 47 through a main picture buffer 44 and a video buffer 24a. 

[0156] As described, according to the second operating example, it is possible to carry 
out process for synthesizing a plurality of camera images, without carrying out 
input/output of image data between a plurality of broadcasting apparatuses or 
repeatedly carrying out A/D conversion and D/A conversion. That is, wasteful 
matter of process in prior art can be eliminated, and there occurs no noise in 
image data due to repeating of A/D conversion and D/A conversion. 

[0157] <Third Operating Example> 

In the third operating example, concrete operation in case of inputting 
(insertion) by operation during broadcasting will be described. 

[0158] In this case, the broadcaster operates an operating button 65 corresponding to 
telop input to switch display in a display region 64 to an operating window for 
telop input. Thereby, process for producing data for telop (Step S3) becomes 
enabled. 

[0159] In the process for producing data for telop, in the operating window for telop 
input, telop input place is selected, for example, by a mouse pointer, letters are 
input in a frame (text box) for telop input displayed on the selected place by 
operating a keyboard 14a, and a button corresponding to Ttelop display J out 
of operating buttons 69 is clicked. Then, in the second switching control (Step 
S6), image data (that is, display data of telop) obtained by the process for 
producing data for telop is selected. 

[0160] In this manner, telop can be inserted into image at real time by editing work 
while executing live streaming broadcasting. 

[0161] As described, according to the third operating example, since telop can be 
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inserted into the image at real time, it is not necessary prepare display data for 
telop in advance to store it, different from the case of prior art (FIG 15), and it 
is possible to carry out insertion of telop in a simple manner.. Further, also in 
the case where telop becomes necessary suddenly, one can correspond thereto 
immediately. 

[0162] <Fourth Operating Example> 

In the fourth operating example, image data produced on the basis of 
designated information (for, example, time information, camera position 
information, score information in game of sports or the like) that is for image 
designation but not image data is synthesized with camera image data. 

[0163] In this case, for example, when T watch display button J Tnot shown J formed 
in an operating screen Gl is clicked, time information is obtained from a time 
portion 15, and image data for time display is produced on the basis of the 
obtained time information, and the image data is synthesized with camera 
image data to output it for broadcasting. 

[0164] <Fifth Operating Example> 

In the fifth operating example, plug-in data (for example, FLASH animation) is 
synthesized with camera image. 

[0165] In this case, when an operating button 67 corresponding to the desired plug-in 
data is clicked, the plug-in data is synthesized with camera image data to output 
it for broadcasting. 

[0166] <Sixth Operating Example> 

In the sixth operating example, a description will be made of the case where 
sprite process is applied to image data and static image data 34 from camera 21. 

[0167] The sprite process is a process wherein for example, specific color of static 
image data 34 is converted into transparent color, and the static image data 34 
and image data from camera 21 are superposed and synthesized so that display 
priority of the static image data 34 is to be upper level. 

[0168] In this case, .for example, as shown in FIG8, process prior to plural camera 
image synthesizing process (Step S2) is different from the case shown in FIG. 
4. 

[0169] That is, image data from camera 21 received in a capture window 41 are 

applied to a third switching control process (Step S21). 
[0170] In the third switching control process, for example, any one of image data, and 

the other image data are selected for storage to the first picture buffer 42, and 
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sprite process (Step S23), respectively. 

[0171] On the other hand, in the fourth switching control process (Step S22), for 
example, any one o out of a plurality of static image data is selected in order to 
apply to the sprite process. 

[0172] In the sprite process (Step S23), for example, the sprite process is applied to 
image data from any one of camera 1, and static image data 34. Image data 
after synthesizing of image data (image data from camera 21) after sprite 
process and static image data 34 are applied to the plural camera image 
synthesizing process (Step S2), which is then synthesized with image data from 
a first picture buffer 42. 

[0173] In the six operating example, clients are able to audit images based on the 
image data applied with the sprite process. 

[0174] <Seventh Operating Example> 

In the seventh operating example, a description will be made of the case where 
while receiving live streaming broadcasting from the other streaming server 
through a network 2, image data f live streaming broadcasting during receiving 
is output to the network 2 for auditory by clients. 

[0175] In this case, in the second switching control (Step S6), image data after 
streaming decoder process by a streaming decoder 46. 

[0176] As a result, image data of live streaming broadcasting received from the other 
streaming server 6, or synthesized image data obtained by synthesizing the 
other image data with the said image data are respectively output (broadcasted) 
to the network 2. 

[0171] According to the seventh operating example, clients are able to audit images 
using image data of the live streaming broadcasting received from the other 
streaming server 6. * 

[0178] <Eighth Operating Example> 

In the case of the eighth operating example, as shown in FIG 9 while receiving 
live streaming broadcasting from a plurality of other streaming server 6 
through a network 2, image data obtained by synthesizing process (streaming 
data synthesizing process: Step S3) for synthesizing image data of a plurality 
of live streaming broadcasting during receiving are output to a network 2 for 
auditory by clients. 

[0179] It is noted, in the streaming data synthesizing process, for example, alpha blend 
process or picture-in picture process is carried out. 



24 



[0180] Further, process (Step S32) for synthesizing the other image data (telop, static 
image, video image data or the like) may be applied to synthesized image data 
obtained by the streaming synthesizing process. 

[0181] Synthesized image data after process of Step S31 or Step S32 are output to the 
network 2 while applying encoding by a streaming encoder 47. 

[0182] As described, according to the eighth operating example, since 
multi-dimensional broadcasting from a plurality of streaming servers 6are 
output to the network 2 for auditory, clients are able to audit multi-dimensional 
broadcasting from a plurality of streaming servers 6. 

[0183] [Second Embodiment] 

In the second embodiment, a description will be made of a synchrony-browser 
function in which link address information of a browser on the broadcaster side 
t output as a script, and a link address of the browser on the clients side is 
designated on the basis of the script of the link address information to thereby 
synchronously switch the link address on the clients side to the broadcaster 
side. 

[0184] FIG 10 is a view showing display on he broadcaster side and the clients side 
during execution of the synchro-browser function. 

[0185] As shown in FIG. 10, on a display screen G2 of a display portion 
12of an editing device 1 on the broadcaster side are displayed a 
browser 91, a mouse pointer 92 within the browser 91, and a 
display region 93 for carrying out (that is, display of image to be broadcasted) 
on the basis of image data produced by any of editing process described in the 
first embodiment. 

[0186] On the other hand, on a display screen G3 of a terminal for auditory4 on the 
clients side are displayed a browser*95, a pointer 96 within the browser 95, and 
a display region 97 for carrying out image display on the basis of image data 
broadcasted, It is noted that display data of the pointer 96 is downloaded, at the 
time of getting access to a server 5 for a home page of the broadcaster, is stored 
and held in the terminal for auditory 4 till the browser 95 is terminated, and is 
used for displaying the pointer 96. 

[0187] Next, it is supposed that operation for switching a link address of the browser 
91 is carried out on the broadcaster side. Then, the editing device 1 converts 
address information of the browser, that is URL (Uniform Resource Locator) to 
script to output it. 
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[0188] Then, the terminal for auditory 4 receives script from the editing device 1 
through a network 2 and a streaming server 3, and converts display of the 
browser 95 to a link address designated by the script. 

[0189] Further, in the second embodiment, position information of a mouse pointer 
(pointer) 92 displayed on a browser on the broadcaster side is output as script, 
and a display position of a pointer 96 on a browser 95 on the auditory side is 
designated on the basis of the script of the position information to thereby 
associate the display position of the pointer 96 on the clients side with the 
pointer 92 on the broadcaster side (synchro-pointer function). 

[0190] That is, the editing device 1 converts position information (coordinate position 
on the browser 91) to script every time when a position of the pointer 92 moves 
on the broadcaster side to output it. 

[0191] Then, the terminal for auditory 4 receives the script from the editing device 1 
through the network 2 and the streaming server 3, and converts the display 
position of the pointer 96 to the position designated by the script (coordinate 
position on the browser 95). 

[0192] Next, these syncro-browser function and synchro-pointer function will be • 
described with reference to a flowchart of FIG. 11. It is noted one shown in FIG 
1 1 is process for which a control portion 1 1 of the editing device 1 is carried 
out. 

[0193] As shown in FIG 11, first, judgment is made whether the synchro-browser 
function is started by operation of the broadcaster (Step S41). 

[0194] In the case where the function is started (YES in Step S41), the coordinate of 
the mouse pointer 92 is converted to the script to perform process for output 
(Step S42, and then link address information of the browser 91 is converted to 
the script for output it (Step S43). 

[0195] In the succeeding step S44, judgment is made whether the synchro-browser 
function is terminated by operation of the broadcaster. 

[0196] In the case where the function is not finished (NO in Step S4), the step moves 
to Step S45. 

[0197] In Step Se45, judgment is made whether the coordinate of a mouse pointer 92 is 
changed, and in the case of judgment in which the coordinate is changed (YES 
in Step S45), process for converting the coordinate of the mouse pointer 92 to 
the script to output it is carried out (Step S46), and the step moves to Step S47. 
On the other hand, in Step S45, in the case of judgment in which the coordinate 
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of the mouse pointer 92 is not changed (NO in Step S45), Step S46 is skipped, 
and the step moves to Step S47. 
[0198] Further, In Step S47, judgment is made whether link address (link address 
information) is changed, and in the case of judgment in which the link address 
is changed (YES in Step 47), process for converting link address information of 
the browser 91 to the script to output it (Step S45), and the step moves to Step 
S44 again. On the other hand, in Step S4, in the case of judgment in which the 
link address is not changed (NO in Step S47), Step S48 is skipped, and the step 
moves to Step S44. 

[0199] Further, in the case where judgment is made in which the synchro-browser is 
finished I Step S44, and in the case where judgment is made in which the 
synchro-browser function is not started in Step S41, the process in FIG 11 is 
finished. 

[0200] According to the second embodiment as described above, since the 
synchro-browser function and synchro-pointer function as described above can 
be realized, for example, presentation, conference or lecture can be suitably 
carried out through the network. At that time, the broadcaster may merely talk 
while touching the browser 91 by the mouse to carry out presentation, 
conference or lecture in a simple manner. 

[0201] Data of small capacity (script of link address information) may merely be 
output for switching display of browser 95 n the clients side, and therefore, data 
capacity handled in the editing device 1 on the broadcaster side can be 
suppressed as small as possible, and broadcasting contents excellent in 
expression can be obtained with less process data amount. 

[0202] In addition, any of broadcasting described in the first embodiment is carried out 
along with the synchro-browser function and synchro-pointer function as 
described above, and therefore, the broadcasting contents can be displayed in a 
display region 97 to enable obtaining broadcasting further excellent in 
expression. For example, in the display region 97, a presenter or a program 
director for conference or lecture is displayed to thereby more easily 
understand presentation, conference or lecture. 

[00203] [Third Embodiment] 

In the third embodiment, a description will be made of an example 
(hand-written function) in which as shown in FIG 12, image data of images 
depicted by operation during broadcasting on a browser 91 on the broadcaster 
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side are output to a network 2 for auditory by clients. 
[0204] In this case, as shown in FIG 12, the broadcaster operates an operating portion, 
for example, such as a mouse 14b during broadcasting to provide a depicted 
image on a browser 91, whereby its depicted image is reflected so that the 
image data is synthesized with, for example, animation data (camera image 
data from camera 21, video image data from a video decoder 45, or image data 
of other streaming broadcasting from a streaming decoder 46) to output it to a 
network 2. 

[0205] As a result, image depicted by operation of the broadcaster is also reflected by 
display on a browser 495 of the auditory terminal 4on the clients side. 

[0206] Next, a flow of process in the case of the third embodiment will be described 
with reference to FIG. 13. 

[0207] Animation data 98a is, as described above, for example, camera image data 
from camera 21, video image data from a video decoder 45, or image data of 
other live streaming broadcasting from a streaming decoder 46. Further, image 
data 98b is image data of an image layer in which depicted image by the 
broadcaster is reflected on display. These image data 98b and animation data 
98a are synthesized by synthesizing process 99. As a result, image data after 
synthesizing is data for displaying to which depicted image depicted by the 
broadcaster is superposed. 

[0208] Such image data after synthesizing is stored in a main picture buffer 44, after 
which it is encoded for streaming broadcasting by a streaming encoder 47 and 
output to a network. 

[0209] The terminal for auditory 4 for receiving image data output as described is able 
to audit broadcasting contents in which depicted image by the broadcaster is 
reflected. * 

[0210] According to the third embodiment as described, the broadcaster performs 
depicted image in a simple manner at real time to enable causing image display 
on the basis of image data of the depicted image to carry out by the terminal for 
auditory 4. Thereby, presentation can be carried out easily through the network 
2. 

[0211] In the above-described embodiments, a description has been made on 
assumption of streaming broadcasting, but for example, the technique for 
outputting image data including plug-in data for broadcasting may be applied, 
not limiting to the live streaming broadcasting, to other broadcasting methods. 
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BRIEF DESCRIPTION OF THE DRAWINGS 
[0212] [FIG 1] A block diagram for explaining a streaming broadcasting method 
according to the embodiment of the present invention. 

[FIG 2] A block diagram showing an editing device and its peripheral devices 
used for a streaming broadcasting method. 

[FIG 3] A view showing main block structure of a control portion provided in 
the editing device. 

[FIG 4] A flowchart for explaining a flow of process with respect to image data 
of editing process carried out by the editing device. 

[FIG 5] A flowchart for explaining a flow of process with respect to voice data 
out of editing process carried out by the editing device. 

[FIG 6] A view showing a screen display example of a display portion of an 
editing device during editing process. 

[FIG 7] A flowchart for explaining a flow of plural camera image synthesizing 
process particularly out of editing process. 

[FIG. 8] A flowchart for explaining an example of process in case of carrying ' 
out sprite process. 

[FIG 9] A flowchart for explaining a flow of process in case of synthesizing 
and outputting live streaming broadcasting receiving from a plurality of other 
streaming servers. 

[FIG. 10] A view showing a screen display example in case of executing 
syncro-browser function and synchro-pointer function. 

[FIG 11] A flowchart for explaining the syncro-browser function and 
synchro-pointer function. 

[FIG 12] A view showing a screen display example in case of executing 
hand-written function. 

[FIG 13] A flowchart for explaining the hand-written function. 

[FIG. 14] A block diagram for explaining a flow of process in a conventional 

live streaming broadcasting. 

[FIG. 15] A block diagram in case of carrying out live broadcasting using a 
number of broadcasting materials in prior art. 

[FIG. 16] A flowchart for explaining a flow of main parts in case of technique 
of FIG 15. 
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[02 1 3] 1 Editing device (live streaming broadcasting device) 

2 Network 

91 Bowser 

95 Browser 

92 Mouse pointer (pointer) 

96 Pointer 

47 Streaming encoder (output means) 

46 Streaming encoder (receiving means) 

1 lb ROM (recording medium) 



